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Abstract Body 

Limit 4 pages single-spaced. 


Background / Context: 

Description of prior research and its intellectual context. 

Children’s social-emotional skills, such as conflict resolution and emotion regulation, have been 
linked to a number of highly regarded academic and social outcomes (e.g., Graziano, Reavis, 
Keane, & Calkins, 2007; Trentacosta & Shaw, 2009). The rich body of research linking social- 
emotional skills and children’s positive outcomes has spurred the design of numerous school- 
based interventions aimed at building those skills (Durlak, Weissberg, Dymnicki, Taylor, & 
Schellinger, 2011). Rigorous evaluations of those interventions typically test direct impacts on 
children’s outcomes. But the intervening mechanisms whereby interventions drive changes in 
learning environments -and ultimately in children’s skills- remain largely unexplored, 
particularly with techniques that support causal conclusions. A better understanding of the 
processes mediating the impacts of interventions on children’s learning environments would 
inform the design of more fine-tuned and cost-effective interventions. 

Purpose / Objective / Research Question / Focus of Study: 

Description of the focus of the research. 

The current study presents preliminary results from a causal test of the theory of change of 
RULER, a universal school-based approach to social and emotional learning (SEE) (Brackett et 
ah, 2011). Specifically, we examine RUEER impacts on the quality of classroom climates, the 
hypothesized setting-level mediating mechanism linking RUEER to improved student outcomes. 
These analyses will provide a more precise understanding of the pathways through which 
RUEER, a specific SEE program, contributes to children’s positive development, and will also 
expand our understanding of the intervening mechanisms underlying effective educational 
interventions more generally. 

Setting: 

Description of the research location. 


Data for this study come from the first randomized control trial (RCT) of The RUEER Approach. 
The RCT took place in 62 primary schools in the Roman Catholic Diocese of Brooklyn in 
Brooklyn and Queens, NY. 

Population / Participants / Subjects: 

Description of the participants in the study: who, how many, key features, or characteristics. 

The effective sample for these analyses includes 24 classrooms (12 in each treatment condition) 
nested in 19 schools. At baseline, schools in the full sample ranged in size from 178 to 656 
students (M = 325.92, SD = 97.06), and the number of students per teacher ranged from 17 to 35. 
Between 5.05 % and 100 % of students were minorities (M= 66.85 %, SD = 32.30 %), and 
between 0 % and 94.67 % received free/reduced lunch (M = 23.34 %, SD = 32.30 %). 40.0 % of 
teachers had worked toward or completed a master’s degree, 36.2 % earned BAs, and 23.8 % did 
not specify. 
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Intervention / Program / Practice: 

Description of the intervention, program, or practice, including details of administration and duration. 

RULER is a multi-year approach available for Kindergarten through eighth grades. It consists of 
a comprehensive professional development program and classroom curricula that leverage 
emotions in the learning environment (Brackett et ah, 201 1). Teachers and other school staff 
receive training on the approach, and develop knowledge and skills to create more organized, 
and intellectually and emotionally supportive learning environments. Students build skills for 
recognizing, expressing, and managing emotions across learning tasks in the context of a 
standard academic curriculum. 

According to RULER theory of change, RUEER is first expected to cause improvements in 
classroom emotional support, which are in turn expected to drive changes in classroom 
organization and instructional support (Hagelskamp, Brackett, Rivers, & Salovey, 2013). 
Improved classroom environments ultimately are expected to improve student outcomes. 

Research Design: 

Description of the research design. 

School principals were introduced to the project in January 2008. Out of 70 principals in 
attendance, 66 volunteered their fifth- and sixth-grade English language arts (EEA) classrooms 
to participate in the evaluation. To ensure a balanced design, participating schools were 
randomly divided into pairs. Each member of the pair was then assigned to the treatment or 
comparison condition using a random number generator. Two treatment and two comparison 
schools dropped out after randomization, for a total of 62 schools, 155 classrooms, and 3,824 
students. 

Baseline comparisons on key school-level variables revealed statistically significant differences 
between conditions on school size (with RUEER schools being smaller in size than comparison 
schools) and classroom organization (with RUEER schools having lower levels of classroom 
organization than comparison schools). These two variables were adjusted for in all analyses. 

Data Collection and Analysis: 

Description of the methods for collecting and analyzing data. 

Data Collection. University-approved informed consent procedures were observed throughout 
the project. Baseline data collection took place in March 2008, and follow up data were collected 
twice per year in the 2008-2009 and 2009-2010 academic years. Observational assessments, 
teacher surveys, and student surveys were collected at each wave. The current study focuses on 
observational assessments of classroom processes, which we describe next. 

Measures. 

Three domains of classroom climate (i.e., emotional support, instructional support, and 
classroom organization) were assessed using the Classroom Observation Scoring System 
(CLASS, Pianta & Hamre, 2009). At each wave of data collection, teachers were asked to 
submit three videotaped classroom sessions. Coders who were blind to treatment condition, and 
who achieved 80% reliability, rated eight- to twenty-minute segments from each videotaped 
session using the CLASS. Two unique coders rated 84.5% of segments. Coders’ ratings were 
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averaged to obtain a single seore for each classroom on each of the three classroom climate 
domains. 

Analysis. Previous SEM path analyses using the same dataset found that significant impacts in 
classroom organization and instructional support after two years of RULER were mediated by 
improvements in classroom emotional support after one year of implementation (Hagelskamp et 
ah, 2013). The current paper employed instrumental variable (IV) techniques (two-stage least 
squares via ivregress in Stata) to conduct a causal test of that mediated path (Gennetian, 
Magnuson, & Morris, 2008). Specifically, random assignment of schools to the intervention was 
used as the instrument to examine whether changes in emotional support after one year of 
intervention, causally mediated changes in classroom organization and instructional supports a 
year later. To account for the clustering of classrooms within schools, we adjusted the standard 
errors of our estimates using the vce(cluster) command in Stata. 

Findings / Results: 

Description of the main findings with specific details. 

Basic descriptive statistics are shown on Table 1. In line with SEM analyses, first stage results 
from the instrumental variable analysis show significant intervention impacts on classroom 
emotional support after one year of intervention {b = .35, p < .05). Second stage results confirm 
that emotional support after a year of implementation has a marginally significant causal impact 
on classroom organization. Unexpectedly, emotional support does not have a statistically 
significant causal impact on instructional support (see Table 2). 

Conclusions: 

Description of conclusions , recommendations, and limitations based on findings. 

Our results partially overlap with prior results that used a non-causal analytic approach. RULER 
impacts on classroom organization were mediated by impacts on emotional support, but the same 
mechanism did not account for impacts on instructional support. 

These findings suggest that creating emotionally supportive environments is a pathway to 
developing more organized and productive classrooms. Teachers who take students’ 
perspectives, and who develop warm and respectful relationships with students, are better able to 
establish classroom rules and routines that can eventually lead to more productive and organized 
learning environments. In contrast to previous findings, more emotionally supportive interactions 
did not account for the improved instructional supports of RULER classrooms. Other aspects of 
RULER, such as an emphasis on making academic content relevant to students’ lives, and 
building students’ vocabulary about emotions, may enhance instructional supports through a 
pathway that is independent from the emotional climate of the classroom. 

Results from these analyses highlight the value of using analytical approaches that allow drawing 
more trustworthy conclusions from non-experimental data. The assumptions underlying IV 
techniques, and the implications of our findings for educational research and practice will be 
discussed at the session. 
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Appendices 

Not included in page count. 
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Appendix B, Tables and Figures 

Not included in page count. 


Table 1 


Bivariate Correlations and Descriptive Statistics for Classroom Variables 





Spring Year 1 



Spring Year 2 





Emotional 

Support 

Classroom 

Organization 

Instructional 

Support 

Emotional 

Support 

Classroom 

Organization 

Instructional Mean 
Support 

Std. Dev. 

Spring Y ear 
1 

Emotional 

Support 

1 





4.605 

0.381 


Classroom 

Organization 

0.6086 

1 




5.458 

0.429 


Instructional 

Support 

0.6822 

0.6921 

1 



2.273 

0.454 

Spring Y ear 
2 

Emotional 

Support 

0.7906 

0.4917 

0.6287 

1 


4.506 

0.391 


Classroom 

Organization 

0.6035 

0.831 

0.4572 

0.5181 

1 

5.486 

0.429 


Instructional 

Support 

0.6437 

0.4796 

0.5966 

0.8435 

0.6039 

1 2.222 

0.492 


Note. All correlations are statistically significant aip < .05. 


Table 2 

Second-Stage Results for Instrumental Variables Model of Classroom Emotional Support, Organization, and Instructional Support 


Dependent variable 

Estimate 

SE 

95% Cl 

full model 

Classroom 

organization 

.55 

.32 

-.08, 1.19 

0.60 

Classroom 
instructional support 

.40 

.48 

-.54, 1.35 

0.36 
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